Polarization-Dependent Metasurface Enables Near-Infrared Dual-Modal Single-Pixel Sensing

Infrared single-pixel sensing with the two most representative modes, bright-field imaging and edge-enhanced imaging, has great application potential in biomedical diagnosis and defect inspection. Building a multifunctional and miniature optical computing device for infrared single-pixel sensing is extremely intriguing. Here, we propose and validate a dual-modal device based on a well-designed metasurface, which enables near-infrared bright-field and edge-enhanced single-pixel imaging. By changing the polarization of the incident beam, these two different modes can be switched. Simulations validate that our device can achieve high-fidelity dual-modal single-pixel sensing at 0.9 μm with certain noise robustness. We also investigate the generalization of our metasurface-based device and validate that different illumination patterns are applied to our device. Moreover, these output images by our device can be efficiently utilized for biomedical image segmentation. We envision this novel device may open a vista in dual-modal infrared single-pixel sensing.


Introduction
Benefiting from the superior detection efficiency and low cost of the single-pixel method [1], infrared single-pixel sensing is emerging as an enabling technology of great technical and scientific interest [2][3][4]. It avoids the use of infrared array sensors with large dark current, low resolution, and high manufacturing cost, but uses a spatial light modulator (SLM) to compress and couple spatial information. It has great potential in various applications, such as defect inspection, biomedical diagnosis [5,6], and remote sensing [7,8]. In these envisioned scenarios, fast and reliable multifunctional sensing plays a vital role, especially when different tasks are needed.
Conventional strategies for different sensing are mainly digital domain computations using integrated circuits at the expense of high-power consumption, low speed, and limited capacity [9]. For better application, optical analog computing is proposed to tailor the field of the incident beam by placing appropriate optical elements in the optical system. It also modulates image information without limiting capacity and allows for parallel operation [10]. However, conventional optical analog computing inevitably increases the complexity and bulk of optical systems, especially when different sensing tasks are needed to be performed [11]. Therefore, developing a multifunctional and miniature optical analog computing device for infrared single-pixel sensing is extremely intriguing.
Metasurfaces, as specially designed two-dimensional optical elements, have become research hotspots. The subwavelength structure of the metasurface can interact with the incident electromagnetic field. These special nanostructures can also exert flexible and large-scale modulation on the amplitude, phase, polarization, and other characteristics of the optical field within the subwavelength thickness range [12,13]. This sudden change in optical parameters breaks the dependence of traditional optical elements on the propagation path. They have great potential in different optical wavefront modulations without changing the components of the optical systems [14][15][16][17][18][19][20]. Meta-designed sensors have shown promise with extremely high sensitivity [21,22]. Moreover, a single metasurface can replace several traditional optical elements, further minimizing the optical systems. These potentials have pushed metasurfaces into optical computing research areas [10,23,24] for conducting some practical tasks, such as optical logic operations [25], optical differentiation operations [26], and optical neural networks [27]. Among these tasks, optical differentiation operations based on computing metasurfaces are mainly used for image edge enhancement. These edge-enhanced images containing textures and morphologies can be used for further sensing tasks. Remarkably, they have been successfully validated for different imaging, including switchable edge-enhanced and bright-field imaging [11,28], tunable edge-enhanced imaging by adding an electric power driver [29,30], etc. In another context, the research on applying the metasurface in the single-pixel imaging domain has attracted extensive attention. Various metasurfaces have been studied for their fascinating roles in conventional single-pixel systems [5,[31][32][33]. So far, a metasurface that can host phase-only and helicity-dependent holograms has been proposed to work as switchable and secret ghost imaging targets. This work builds the first bridge between the metasurface hologram and single-pixel imaging [5]. A metasurface can also be a prototype SLM for high-frame-rate single-pixel imaging or simplifying the single-pixel imaging system [31,32]. These works provide new applications for single-pixel imaging. More interestingly, a novel optical encryption scheme has also been proposed depending on the combination of metasurface hologram and single-pixel imaging technology [33]. The metasurface for dualmodal single-pixel sensing is valuable but has not received sufficient attention. Regardless, the development of these metasurface-based devices brings great hope to our research.
Here, we report a dual-modal metasurface-based device for near-infrared single-pixel sensing, which enables infrared and edge-enhanced imaging as requested. The dual-modal device is mainly achieved by a designed metasurface, which can function as different spatial filters just by rotating the polarizer in this device. Specifically, when the polarization of the incident beam is y-linear polarization (YLP), the output mode is high-fidelity bright-field imaging. Similarly, when the polarization of the incident beam is x-linear polarization (XLP), the output mode is high-fidelity edge-enhanced imaging. Additionally, this device is suitable for various illumination patterns, and the same illumination pattern can be used in dual modes. Results show that this device can achieve high-fidelity dual-modal single-pixel sensing at 0.9 µm with certain robustness. Moreover, the output images of our device can be efficiently utilized for further computer vision tasks, such as biomedical image segmentation. Figure 1 is the framework of the proposed device for near-infrared dual-modal singlepixel sensing. This device consists of a 4f imaging system embedded with a polarizationdependent metasurface and classical single-pixel imaging optical elements, which can efficiently obtain bright-field or edge-enhanced images. We can switch these two functions only by rotating the polarizer. Specifically, when the polarization of the incident beam is YLP, and the SLM projects Fourier basis patterns on sequence, the output field is a reconstructed high-fidelity bright-field image. Similarly, when the polarization of the incident beam is XLP, and the SLM projects the same illumination patterns, the output field is a reconstructed high-fidelity edge-enhanced image. In addition, this device is suitable for various illumination patterns, and the images obtained by our device can be used for further computer vision tasks, which we will show later.  To explain the feasibility of our proposed device, we analyze the whole processing and relevant principle in detail. In this system, the target scene is modulated to an XLP or YLP incident beam E in (x, y) by the rotatable polarizer. x and y are, respectively, xand y-direction coordinates in the input or output image plane. Then spatial filtering is achieved by optical computing. It is contributed by the 4f imaging system embedded with a polarization-dependent metasurface. The computed field distribution E M can be written as:

Principle of the Device
where F represents a 2D spatial Fourier transformation, F −1 represents a 2D inverse spatial Fourier transformation, and H( f x , f y ) is the optical spectrum transfer function, which is contributed by our metasurface. f x and f y are, respectively, uand v-direction spatial frequency coordinates in the Fourier plane. Here, two different H( f x , f y ) are designed for dual-modal single-pixel sensing. When bright-field imaging mode is chosen, the H( f x , f y ) is equal to a constant. When edge-enhanced imaging mode is chosen, we utilize spiral phase contrast imaging based on the vortex beam [28] to design the H( f x , f y ). In this regard, H( f x , f y ) can be written as: where H x f x , f y and H y f x , f y are optical spectrum transfer functions for edge-enhanced (XLP) and bright-field imaging (YLP), respectively, Our metasurface required phase profiles are Ψ x = φ + C 1 and Ψ y = C 2 , which are, respectively, under the illumination of an XLP and YLP incident beam. Next, the E M (x, y) field distributions for dual modes are both modulated by the corresponding patterns [34] projected onto the SLM. The obtained inner product D j between patterns P j (x, y) and E M (x, y) is measured by the single-pixel detector, which can be written as: Accordingly, we finish the acquisition of the modulated target scene in dual modes. Then, the Alternating Direction Method of Multipliers (ADMM) framework is utilized to reconstruct the target image O. We introduce an auxiliary parameter Q to build the objective function into: where A ∈ R m×n denotes the modulation matrix (m modulation patterns, and each pattern consists of n pixels), and D ∈ R m×1 is the measurement vector. In addition, TV(O) is the total variation regularization term. The following distributed sub-problems can solve the minimization in Equation (5), where the superscript k represents the iteration number, and ρ represents the hyperparameter. The sub-problem has a closed-form solution: where α represents the hyper-parameter. When the above equation converges, we can reconstruct the target's bright-field or edge-enhanced images as requested.
In addition, we want to emphasize that different patterns correspond to different modulations, the main ones being Fourier, Hadamard, and random modulations. We will describe these modulations in detail next. It should be noted that this work mainly uses Fourier modulation, but this does not affect the generalization of our method. We also prove the generalization of the method in Section 3.3.

Fourier Modulation
Fourier modulation projects sinusoidal patterns to the target scene and captures the one-dimensional signal with the bucket detector [34]. The Fourier basic pattern P can be expressed as: where φ represents the initial phase. We use three-step phase shifting to sample images. Each coefficient in the Fourier space F( f x , f y ) is derived using three sinusoidal patterns with the same spatial frequency and different initial phases: where D 0 , D 2π/3 , and D 4π/3 are the measurements corresponding to the illumination patterns of P(x, y, 0), P(x, y, 2π/3), and P(x, y, 4π/3), respectively. Because of the conjugate symmetric feature of the Fourier spectrum, we only need to measure the upper half of the Fourier coefficients.
Then, the objective function can be transformed into: The problems can be solved like Equations (6) and (7).

Hadamard Modulation
Hadamard modulation is based on Hadamard transform. By applying an inverse Hadamard transform to a delta function δ H (u, v), the Hadamard transform pattern P H (x, y) can be obtained [3,35,36]: where H and H −1 {} denote a Hadamard transform and an inverse Hadamard transform, respectively.
Each Hadamard coefficient H(u, v) is acquired by two measurements. They are one measurement corresponding to a Hadamard basis pattern P H (x, y) and one measurement corresponding to an inverse pattern [1 − P H (x, y)]. The coefficient H(u, v) is obtained by using the two corresponding measurements: where D +1 and D −1 are measurements corresponding to the illuminations of P H (x, y) and Then, the objective function can be transformed into: The problems can be solved like Equations (6) and (7).

Binary Random Modulation
The binary random pattern P j is generated by a binary pseudo-random number generator [37]. This modulation method can realize the projection consistent with the speed of the SLM, because the number of bits is only one. It greatly increases the modulation speed. The objective function can be solved directly by referring to Equations (5)-(7).

Design of Metasurface
To realize the proposed dual-modal metasurface-based device, we designed the dualfunctional metasurface mentioned in Section 2. When the polarization of the incident beam is XLP, the designed metasurface should work as an edge filter in the Fourier plane. When the polarization of the incident beam is YLP, the designed metasurface should work as a bright-field filter in the Fourier plane. We used the polarization-dependent propagation phase of the nanobrick to achieve the two different phase profiles mentioned above, Ψ x and Ψ y . The simulation results were calculated through the software-FDTD solutions, where periodic boundary layers are used in the x and y directions, and a perfectly matching layer is used in the z direction. The mesh accuracy is equal to 2. The distance of the monitor from the nanobricks is 3 µm. Dispersion is included in the material data. The plane-wave sources are utilized. Figure 2a displays a side view of a specially designed unit cell consisting of a silicon nanobrick sitting on a fused silica substrate. These nanobricks are periodically arranged with a fixed square lattice constant P = 360 nm and a height H = 500 nm. The propagation phase can basically cover 0 ∼ 2π by changing the major semi-axis a and minor semi-axis b of the nanobrick. The schematic for computing the transmission coefficients (t x , t y ) and phase shifts (δ x , δ y ) is shown in Figure 2b.
The simulated phase distribution δ x for an XLP incident beam as a function of a and b is shown in Figure 2c, and the corresponding transmission coefficient distribution t x is exhibited in Figure 2d. Similarly, the simulated phase distribution δ y for a YLP incident beam is shown in Figure 2e, and the corresponding transmission coefficient distribution t y is exhibited in Figure 2f. Accordingly, we chose an appropriate size (a, b) of the nanobrick in theory to obtain phase combination δ x , δ y , which is equal to Ψ x , Ψ y . We chose 16 discrete phases in the range of 0 ∼ 2π, and picked 16 corresponding nanobricks based on the minimum phase difference with the phase in our library and the maximum transmittance coefficient. The 16 selected nanobricks for constructing the metasurface mentioned in Section 2 are shown in Table 1. The designed metasurface consists of 109 nanobricks along both x and y directions. The working wavelength of the incident laser is 0.9 µm. To illustrate that the selected nanobricks are appropriate, we calculated the ideal and real phase profile distributions of the designed metasurface. Figure 3a shows the ideal phase profile distribution Ψ x of the designed metasurface under the illumination of the XLP. Figure 3b shows the ideal phase profile distribution Ψ y of the designed metasurface when illuminated by the YLP. Figure 3c,d display our designed metasurface's real phase profile patterns under the illumination of the XLP or YLP incident beam. The phase distributions in Figure 3a,c are largely similar, albeit not entirely consistent, due to the selection of only 16 different nanobricks. However, these deviations do not significantly impact the function, as the spiral characteristic of the designed phase profile is well maintained. Similarly, the phase distributions in Figure 3b,d exhibit some inconsistencies due to the limitations of the nanobricks' types, but these deviations are within acceptable tolerances. Other simulated results are shown in Figure 4. Respectively, Figure 4a,b show the far field intensity and phase distribution of the designed metasurface when illuminated by the XLP plane wave. Figure 4c,d, respectively, display the far field intensity and phase distribution of this metasurface when illuminated by the YLP plane wave. The doughnut-shaped intensity distribution is transformed into the Gaussian intensity distribution by altering the polarization of the incident beam. The phase pattern converts from a spiral-like distribution to an approximate constant distribution in the central area when switching the polarization of the incident beam. These figures show that our polarization-dependent metasurface is well designed.

Full-Process Simulations
To validate the implementability of our proposed device, we simulate the whole processing mentioned in Section 2. Based on MATLAB and FDTD-solutions, we first simulate the optical computing in Equation (1). This modulation is contributed by a 4f system embedded in our designed metasurface. Its computing result is shown in Figure 5. The input images are, respectively, a "BIT" plus cardiogram image, an infrared image [38], and USAF. Specifically, when the polarization of the incident beam is XLP, the designed metasurface should work as an edge filter in the Fourier plane. When the polarization of the incident beam is YLP, the designed metasurface should work as a bright-field filter in the Fourier plane. These bright-field filtering outputs are generally fainter than the ground truth but acceptable when illuminated by the YLP incident beam. All edge information after filtering is obvious when illuminated by the XLP incident beam. Moreover, Figure 5c shows details of the orange line on USAF, which maintains sharp textures and enhances the edge information. It could be concluded that optical computing based on the designed metasurface can efficiently achieve a dynamic switch between bright-field filtering and edge-enhanced filtering at 0.9 µm, by changing the polarization of the incident beam.
Next, we couple these intermediate results with the corresponding Fourier basis patterns and reconstruct the relevant images [256,256] in dual modes. We obtain the final results at different sampling ratios and compare them with those of conventional single-pixel imaging [34], as shown in Figure 6. Imaging results are generally fainter than conventional single-pixel imaging, but overall details are still well maintained. As the sampling ratio rises, the structures and details are recovered more clearly. The details of the orange line on USAF show that the device could maintain sharp textures and edge information at a low sampling ratio. It could be concluded that our proposed dual-modal metasurface-based device can indeed achieve high-fidelity dual-modal sensing.  Figure 6. The proposed dual-modal single-pixel sensing of our device and conventional single-pixel imaging [34].

Generalization Analysis
In addition, we also investigate the generalization of our dual-modal device in modulation. As shown in Figure 7, we applied Hadamard patterns [3,35,36], Fourier patterns [34], and random binary patterns [37] to couple the target information in two different modes. We obtained 6554 measurements of the cameraman image [128,128] in the above modulations. From these figures, we can find that all of these bright-field and edge-enhanced images can be well reconstructed. Although the random method is slightly worse than the other modulations, these results still illustrate that various patterns are applied to our device.

Robustness Analysis
In practical scenarios, various factors such as dark current and thermal noise usually affect the imaging quality. To further perform an actual scene, we simulated the whole acquisition and reconstruction processing under different noise levels. Gaussian white noise was added to the one-dimensional measurements. The sampling ratio was 10%. As shown in Figure 8, the reconstructed images basically maintain most details under different noise levels. Edge-enhanced images are more sensitive to noise than bright-field images because the frequencies of edge-enhanced images are concentrated on high-frequency bands. In brief, despite the relatively high noise level, our device can still obtain acceptable edge-enhanced or bright-field images. Therefore, we think our proposed device has certain noise robustness.

Bright-field imaging
Edge-enhanced imaging In addition, random errors may arise during the processing of the metasurface due to unstable factors such as the environment. Robustness analysis is important as these random errors may result in changes in or even the deterioration of the experimental results. Simulating the process can help analyze the robustness and set reasonable accuracy requirements for the actual processing of the metasurface. During the fabrication, the central position of the nanobrick is relatively accurate, while its shape is not easily accurate and has a relatively great influence on phase modulation. Therefore, we kept the height of the nanobricks constant and added random errors mainly to the major and minor axes of each nanobrick. Three simulations were conducted with random errors ranging from −5 to 0 nm, 0, and 0 to 5 nm for each simulation to observe the results under different error ranges. The sampling ratio was 10%. The bright-field image obtained when the polarization of the incident beam is YLP is shown in Figure 9a, and the edge-enhanced image obtained when the incident beam is XLP is shown in Figure 9b. Our findings show that the simulation results under different ranges of random errors are various, but it is not very obvious. The overall image quality is still acceptable, indicating that our metasurface design can tolerate certain processing errors.

Biomedical Applications
In biomedical diagnosis, segmenting cell substructures allows for the analysis of clinical parameters related to volume and shape [39]. Bright-field imaging is high in redundancy but beneficial for complex segmentation tasks. However, medical images usually have low contrast and complex microstructure distributions, so edge image acquisition is one of the critical technologies in medical image processing. For segmentation, edge-enhanced images containing morphologies [40,41] are used to confirm the target's boundaries. Many researchers have considered the meaning of edge enhancement in medical segmentation, which has effectively improved the segmentation accuracy through methods such as Edge Attention Network (ET-Net) [40], and KiU-Net [41]. These all demonstrate the important role of edge information in medical segmentation. Our device containing optical computing can directly output these bright-field and edge-enhanced images. It can work as the front end of neural networks for extracting features and directly provides edge information for network training, thereby reducing training parameters and computation.
To validate that our device can be smoothly combined with the neural network, we used biomedical images modulated by our device to train the classical Unet [42] for cell segmentation. The initial dataset is from serial section transmission electron microscopy of the Drosophila first instar larva ventral nerve cord (VNC) [43]. It contains 30 training images, whose sizes are [512,512]. We expanded the dataset to 1200 images by randomly cropping the original training images into [128, 128] pixels. In this dataset, the corresponding ground truth segmentation results are manually sketched, where the cells and membranes are marked in white and black, respectively. The framework of Unet is shown in Figure 10, which has a contracting path to capture context and a symmetric expanding path that enables precise localization. We used the normalization initialization method with the bias initialized to 0 and used the Adam solver for gradient optimization. The weight decay was 1 × 10 −3 and was decreased by 0.1 at 50 epochs and 350 epochs. We utilized Cross-Entropy loss, BCE loss, and Dice loss to train Unet. We obtained segmentation results of cells and membranes when we input these corresponding bright-field and edge-enhanced images into the trained Unet. Figure 10 validates that the outputs of our device could be efficiently applied to extract target features. We believe that the combination of bright-field and edgeenhanced images provides a low-power consumption approach for medical segmentation under a rationally designed network.

Conclusions
In summary, we report a near-infrared dual-modal single-pixel sensing device based on a polarization-dependent metasurface, which realizes switchable edge-enhanced imaging and bright-field imaging. Results show that it can achieve high-fidelity dual-modal singlepixel sensing at 0.9 µm and has certain noise robustness. We explored the proposed device's potential in biomedical image analysis. The advantages of the dual-modal device lie in three aspects. First, this device realizes dual-modal sensing in a simple optical system. By rotating a polarizer to change the polarization of the incident beam, the designed polarization-dependent metasurface can function as different filters for different sensing tasks. Second, this novel device could be applied to all illumination patterns, which differs from the existing edge single-pixel methods. It maintains generalization ability on different illumination patterns. The same illumination patterns can be utilized for dual modes, even different sensing tasks. These advantages will not limit further optimization for highaccuracy imaging. Third, the optical analog computing based on our designed metasurface can process target scenes with high speed and low-power consumption. Moreover, these kinds of devices containing optical computing have widely worked as the front end of neural networks for extracting required information from high-redundancy target scenes. Therefore, we can envision that our device, combined with the neural network, can pave a new path for further intriguing sensing tasks.